The main objective of this article is to improve the accuracy of Mamdani fuzzy rule-based classification systems. Although these systems tend to perform successfully with respect to interpretability, they suffer from rigid pattern space partitioning. Therefore, a new hierarchical fuzzy rule-based classifier based on binary-tree decomposition is proposed here to develop a more flexible pattern space partitioning. The decomposition process is controlled by fuzzy entropy of each partition. Final rule sets obtained by this proposed method are pruned to overcome the over fitting problem. The performance of this method is compared with some fuzzy and non-fuzzy classification methods on a set of bench mark classification tasks. The experimental results indicate a good performance of the proposed algorithm.